Papers with learning algorithm
Spa: On the Sparsity of Virtual Adversarial Training for Dependency Parsing (2022.findings-aacl)
Copied to clipboard
| Challenge: | Virtual adversarial training (VAT) is a powerful approach to improving robustness and performance, leveraging both labeled and unlabeled data to compensate for the scarcity of labeles. |
| Approach: | They propose a Sparse Parse Adjustment algorithm which combines VAT and a graph-based dependency parsing model in an exact computational manner and enhances the dependency parsed with controllable and adjustable sparsity. |
| Outcome: | Empirical results show that the proposed algorithm outperforms other methods without sparsity regularization. |
Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have extensive world knowledge, but often generate inaccurate geospatial knowledge. |
| Approach: | They propose a framework for evaluation of large language models to mitigate hallucinations . they use Kahneman-Tversky Optimization to align LLMs with their reality . |
| Outcome: | The proposed evaluation framework uncovers hallucinations in 20 advanced LLMs. |
Systematic Generalization on gSCAN with Language Conditioned Embedding (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing deep learning models fail when the test set is systematically different from the training data. |
| Approach: | They propose a method that explicitly models the relations between objects in their contexts while learning their representations. |
| Outcome: | The proposed model outperforms the baseline model and reaches state-of-the-art performance on grounded SCAN (gSCAN), a grounded natural language navigation dataset. |
EDU-level Extractive Summarization with Varying Summary Lengths (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on extractive summarization use finer-grained elementary discourse units . few studies exploited finer grained EDUs with little analysis and justification for the extractive unit selection . |
| Approach: | They propose an extractive model with Varying summary lengths that extracts fixed top-k salient sentences from the document as a summary. |
| Outcome: | The proposed model performs better on ROUGE scores than state-of-the-art models. |
Influence Scores at Scale for Efficient Language Data Sampling (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that ML models can be fine-tuned on as much data as possible without degradation in performance metrics. |
| Approach: | They evaluate the applicability of influence scores in language classification tasks by random sampling and stress-testing one of the scores. |
| Outcome: | The proposed model can be fine-tuned on 50% of the original data without degradation in performance metrics. |
Active Imitation Learning with Noisy Guidance (2020.acl-main)
Copied to clipboard
| Challenge: | Structured prediction methods learn models to map inputs to complex outputs with internal dependencies. |
| Approach: | They propose an algorithm that mimics an expert's choice at any queried state . they apply LEAQI to three sequence labelling tasks to reduce query costs . |
| Outcome: | The proposed algorithm shows better accuracies over a passive approach. |
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Learning to Learn Semantic Parsers from Natural Language Supervision (D18-1)
Copied to clipboard
| Challenge: | Existing logical forms require a user to be familiar with the underlying structure to learn a semantic parser. |
| Approach: | They propose a method for training semantic parsers from natural language feedback . they use natural language inputs to parse feedback to leverage it as a form of supervision . |
| Outcome: | The proposed algorithm learns a semantic parser from users’ corrections expressed in natural language. |
Minimizing Annotation Effort via Max-Volume Spectral Sampling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Spectral sampling strategies that minimize the number of annotations required to train a model are proposed. |
| Approach: | They propose a method that maximizes the amount of information useful for the learning algorithm by minimizing redundancy of samples in the selection. |
| Outcome: | The proposed method maximizes the amount of information useful for the learning algorithm or minimizes redundancy of samples in the selection. |
OIE@OIA: an Adaptable and Efficient Open Information Extraction Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Different Open Information Extraction (OIE) tasks require different types of information. |
| Approach: | They propose to adapt an OIE Graph to different OIE tasks with simple rules . they implement an end-to-end OIA generator and make it open-accessible . |
| Outcome: | The proposed system achieves new SOTA performance on three popular OIE tasks. |
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation (2023.acl-long)
Copied to clipboard
Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Lianhui Qin, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, Yejin Choi
| Challenge: | Empirical results suggest that scale is not the only way to build commonsense capabilities. |
| Approach: | They propose a commonsense distillation framework that can achieve a competitive level of commonsensing without relying on the benefits of scale. |
| Outcome: | The proposed framework breaks the dependence on the extreme-scale teacher model with two innovations: (1) the novel adaptation of NeuroLogic Decoding to enhance the generation quality of the weak, off-the-shelf language models, and (2) self-imitation learning to iteratively learn from the model’s own enhanced commonsense acquisition capabilities. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
Can Transformers Learn n-gram Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work has tested transformers' ability to represent formal languages, but language models are not classifiers of strings but rather distributions over them. |
| Approach: | They relate transformers' ability to learn random n-gram language models to ngram language model (LM) they find add- smoothing outperforms transformers on the former, while transformers perform better on the latter . |
| Outcome: | The proposed models outperform classical methods designed to learn n-gram LMs, while transformers perform better on the latter. |
Learning with Different Amounts of Annotation: From Zero to Many Labels (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of annotator agreement can hinder training of NLP systems . we propose a learning algorithm that can learn from training examples with zero, one, or multiple labels. |
| Approach: | They propose an annotation distribution scheme that assigns multiple labels to training examples . they propose a learning algorithm that can learn from training examples with different amount of annotation . |
| Outcome: | The proposed method achieves consistent gains in two tasks, suggesting distributing labels unevenly among training examples can be beneficial for many NLP tasks. |
What Do Language Models Learn in Context? The Structured Task Hypothesis. (2024.acl-long)
Copied to clipboard
| Challenge: | Pre-trained large language models have exhibited an impressive ability to learn in context across various domains, e.g., code generation, education, medicine and even medicine. |
| Approach: | They taxonomize existing candidate theories into three competing hypotheses that explain LLMs’ ability to learn in context. |
| Outcome: | The proposed model can learn a task from in-context examples presented in a demonstration and generalize it to the prompt. |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
Is Child-Directed Speech Effective Training Data for Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | High-performing language models are typically trained on hundreds of billions of words, but human learners use language fluently after far less training data. |
| Approach: | They train GPT-2 and RoBERTa models on 29M words of English child-directed speech and a new matched, synthetic dataset. |
| Outcome: | The proposed models show that child language input is not valuable for training language models. |